Last month the American Psychological Association released a report titled “American Psychological Association Task Force on Violent Media Technical Report on the Review of the Violent Video Game Literature.” It “confirmed” the relationship between playing violent video games and aggression.
This was truly disappointing.
The body of literature on violent video games has long been evolving in complexity and moving beyond a “violent video games = aggression model.” For example, newer research does not assume a one-to-one relationship between playing violent video games and aggression (as is frequently published and reported). Instead, newer studies have been examining related factors that might more fully explain the relationship previous studies have found. For example, Adachi and Willoughby examined the role competition plays in eliciting aggression. That study found that violent content without competition had no effect.
More than 200 psychologists, media experts, and criminologists wrote an open letter to the APA voicing their objections to the task force’s report. These objections included biased members on the task force, inconsistent selection of studies, and exclusion of null (no finding) reports.
The Most Critical Flaw
For me, the meta-analytic approach used by the task force is the most critical and fundamental flaw with the report. A colleague of mine put it best when he said, “It’s a meta-analysis of biased research.”
When conducting a meta-analysis, you collect as many studies as you can on the topic. Then you compare them to a list of criteria set forth by the researchers, convert the different metrics used to measure effect, and compare the results. It’s kind of like finding the average. But instead of a list of numbers it’s an average of the results of the studies analyzed. And this is a huge problem in the world of video game research because so much of the research on video games has been ill-constructed. This is in addition to a profession-wide problem of biased publications. Publishers tend to publish only those studies or findings that concur with popular opinion or have some kind of marked result. Null results (e.g.,studies that find no relationship between violence and video games) often go unpublished even though they are just as important.
To understand just how flawed the task force’s report is, you have to go to the source material. The task force reported collecting 170 peer-reviewed articles. Only 68 of those met all aspects of the task force-generated criteria list. Those 68 studies were assessed for “sufficient utility,” producing 31 studies for analysis. Digging even deeper, the task force stated only 18 of the remaining studies had detailed enough statistics (effect sizes, for those interested) to compare.
Consider that for just a moment. If you search Google Scholar for “violent video games” between 2009 and 2013 (the time frame used in the study), you get more than 15,000 results. Add on the peer-review filter, and 1,500 results are returned. Those 31 studies are about 2% of the peer-reviewed literature during that time frame. And 18 studies is 1.2% of available studies.
Then there’s the studies themselves. When I originally was planning this article, I intended to go through the 18 different evaluated studies and critique each one. However, this proved to be too large a task. In the interest of time, and my sanity, the following is a critique—in the plainest terms I can use—of one of the 18 studies. Full disclosure, this was the very first study I read, and in no way is this critique meant to single out the authors.
Just One Example: An Analysis of a Study
Article: “Violent Video Games Cause an Increase in Aggression Long After the Game has Been Turned Off”
Authors: Brad J. Bushman and Bryan Gibson
Overview: This study featured 126 college students, 69 of which were male. Participants were randomly assigned to play one of the following games: Mortal Kombat vs. DC Universe; Resistance: Fall of Man; Resident Evil 5; Guitar Hero; Gran Tourismo 5; or Shaun White Snowboarding. The first three were labeled as violent and the second three were labeled as nonviolent. A coin flip decided if a participant played a violent or nonviolent game. Participants played for 20 minutes and then rated on a scale from 1 (strongly disagree) to 10 (strongly agree) on how “absorbing, action-packed, arousing, boring, enjoyable, entertaining, exciting, frustrating, fun, involving, stimulating, addicting, and violent” they experienced the game to be. Participants also wrote down their three favorite video games and were reminded to return the following day. Before leaving, half of the players who played the violent games and half the players who played the nonviolent games were asked to ruminate about the game.
“In the next 24 hours, think about your play of the game, and try to identify ways your game play could improve when you play again.”
The following day, participants were allotted three minutes to write down what they had thought about in the past 24 hours. They then completed a competitive reaction-time test in which the winner could punish the loser with a loud, painful noise.
Results:
- Games in the “violent” group were rated as being more violent than games in the “non-violent group” but no other interactions were found. That is, the games did not appear to differ in terms of enjoyableness, fun, excitement, etc.
- Habitual exposure to violent video games was assessed by counting how many “M” rated games the participants listed in their top three video games.
- Those who were asked to ruminate thought about the video game more than those who were not asked to ruminate.
- Men who played the violent video game AND ruminated were “more aggressive” than men who did not ruminate. Men who played the nonviolent game showed no increase in aggression regardless of the rumination condition.
- No effect was found in women. That is, women who played a violent game and ruminated did not show increased aggression compared to women who played the violent video game but did not ruminate.
The Claim: Playing a violent video game for 20 minutes and then asking a person to think about that game for 24 hours stimulated aggression in men if they ruminated about the violent content.
This last part, in bold, is crucial to understand. The title of this article is “Violent Video Games Cause an Increase in Aggression Long After the Game has Been Turned Off.” But what the authors 
are saying is men who play violent video games demonstrate an increase in aggression, but only if told to ruminate on the subject. This is of critical importance because rumination is a compulsive re-living of distress. It is a common symptom in psychological disorders including depression, anxiety, and obsessive-compulsive disorder. The researchers, in effect, induced a psychologically unhealthy state on their participants.
Wait, you might say, the group that ruminated over the nonviolent game didn’t experience the same increase in aggression. I would applaud you for your keen observation but encourage you to examine the games that were played.
For those who are unfamiliar:
- Mortal Kombat vs. DC Universe is a fighting game.
- Resistance: Fall of Man is a war game.
- Resident Evil is a horror game.
- Gran Tourismo 5 is a racing game.
- Guitar Hero is a musical rhythm game.
- Shaun White Snowboarding is a sports game.
Think about worst-case, distressing outcomes across these games and it’s not surprising the violent group experienced a reaction. The distress in the three violent games is life and death, especially in Resident Evil, a horror game. Resident Evil is specifically designed to activate our fear centers. Meanwhile, the worst thing that can happen in the three non-violent games is that you fail. The violent games rely on “twitch” reactions, a rapid knee-jerk type response style common in shooters. Get them before they get you. The nonviolent games do not have a twitch style of gameplay and require foresight and planning rather than knee-jerk reactions. It is not beyond the realm of possibility that men who were instructed to pathologically think about life-and-death situations experienced an effect while men who were instructed to think pathologically about recreation did not.
Another issue with the game selection is how different each of the games is. Previous studies have matched games based on the graphics, violence, pace of action, etc. to enable a more accurate comparison between the titles. The violent games might all contain violence but have very little else in common.
- Mortal Kombat vs. DC Universe: single-player 3D fighting game
- Resistance: Fall of Man: first-person shooter, single player
- Resident Evil 5: third-person action/horror game, single player
Looking beyond the games themselves, there are some very important questions not addressed by the authors. For example, there does not seem to be any data pertaining to the participants’ previous experience with video games or the PlayStation 3, on which the games were played. Experience with a title has been shown to influence how a person reacts to it. Participants played for only 20 minutes, which is also problematic. Research has demonstrated that there is a spike in aggression within the first 15-20 minutes. This spike is likely due learning things like level design, control settings, and game mechanics, as well as obtaining objectives, but it dissipates over time.
There’s also the issue of measuring aggression. In this particular case, whether or not a participant blasted another person with a loud noise was the metric for studying aggression.
It’s also important to keep in mind 44% of participants (all the female participants) experienced no increase in aggression regardless of the group they were in. The researchers explain this as “women don’t like violence in games.” However this finding likely says more about social norms—women shouldn’t be aggressive, women shouldn’t blast loud noises at people—than violence in video games.
You can do lots of nonviolent activities in Skyrim.
Yet another issue that weakens this article is the researchers’ decision to use the M rating of favorite video games as a measure of violence habituation. Identifying a game as your favorite does not mean that you are currently playing that game, or that the game is in any way similar to the games used in the study (other than the M rating). For example, Elder Scrolls V: Skyrim is a single-player role- playing game in an enormous fantasy world. You can smith armor, become a wizard, slay dragons, save the world, make potions, cook stew, help reunite loved ones, liberate people from oppression, and so much more. Leisure Suit Larry is also rated M. It’s a graphic adventure about Larry trying to get women to sleep with him.
Not all Mature-rated games are created equal.
The cherry on top of this sundae is the APA report itself explicitly stating (on page 22) that ESRB ratings are an unreliable means of defining violent content and saying, “This definitional problem has undoubtedly colored the research findings of this field.”
This one study makes up 3% of those used in the analysis. That may not seem like much, but remember the studies used in this analysis comprised only 1.2% of the available research. It’s also important to note that Bushman, an author of this study, was also lead or coauthor on three other studies used in the meta-analysis. That means his interpretation of data on video games comprised roughly 13% of the meta-analytic study. Pile on the extensive list of problems as outlined by Keezy Young on Pixelkin, and you’ll want to take a deep breath as you ponder this report as the APA’s current disposition on violence in video games.
